Added benefits of computer-assisted analysis of Hematoxylin-Eosin stained breast histopathological digital slides

This thesis aims at determining if computer-assisted analysis can be used to better understand pathologists’ perception of mitotic figures on Hematoxylin-Eosin (HE) stained breast histopathological digital slides. It also explores the feasibility of reproducible histologic nuclear atypia scoring by incorporating computer-assisted analysis to cytological scores given by a pathologist. In addition, this thesis investigates the possibility of computer-assisted diagnosis for categorizing HE breast images into different subtypes of cancer or benign masses. In the first study, a data set of 453 mitoses and 265 miscounted non-mitoses within breast cancer digital slides were considered. Different features were extracted from the objects in different channels of eight colour spaces. The findings from the first research study suggested that computer-aided image analysis can provide a better understanding of image-related features related to discrepancies among pathologists in recognition of mitoses. Two tasks done routinely by the pathologists are making diagnosis and grading the breast cancer. In the second study, a new tool for reproducible nuclear atypia scoring in breast cancer histological images was proposed. The third study proposed and tested MuDeRN (MUlti-category classification of breast histopathological image using DEep Residual Networks), which is a framework for classifying hematoxylin-eosin stained breast digital slides either as benign or cancer, and then categorizing cancer and benign cases into four different subtypes each. The studies indicated that computer-assisted analysis can aid in both nuclear grading (COMPASS) and breast cancer diagnosis (MuDeRN). The results could be used to improve current status of breast cancer prognosis estimation through reducing the inter-pathologist disagreement in counting mitotic figures and reproducible nuclear grading. It can also improve providing a second opinion to the pathologist for making a diagnosis

Radiologists can detect the ‘gist’ of breast cancer before any overt signs of cancer appear

Radiologists can detect abnormality in mammograms at above-chance levels after a momentary glimpse of an image. The study investigated this instantaneous perception of an abnormality, known as a “gist” response, when 23 radiologists viewed prior mammograms of women that were reported as normal, but later diagnosed with breast cancer at subsequent screening. Five categories of cases were included: current cancer-containing mammograms, current mammograms of the normal breast contralateral to the cancer, prior mammograms of normal cases, prior mammograms with visible cancer signs in a breast from women who were initially reported as normal, but later diagnosed with breast cancer at subsequent screening in the same breast, and prior mammograms without any visible cancer signs from women labelled as initially normal but subsequently diagnosed with cancer. Our findings suggest that readers can distinguish patients who were diagnosed with cancer, from individuals without breast cancer (normal category), at above-chance levels based on a half-second glimpse of the mammogram even before any lesion becomes visible on the mammogram. Although 20 of the 23 radiologists demonstrated this ability, radiologists’ abilities for perceiving the gist of the abnormal varied between the readers and appeared to be linked to expertise. These results could have implications for identifying women of higher than average risk of a future malignancy event, thus impacting upon tailored screening strategies

Detection of the abnormal GIST in the prior mammograms even with no overt sign of breast cancer

Can radiologists distinguish prior mammograms with no overt signs of cancer from women who were later diagnosed with breast cancer from the prior mammograms of women reported as normal and subsequently confirmed to be cancerfree? Twenty-three radiologists and breast physicians viewed 200 craniocaudial mammograms for a half-second and rated whether the woman would be recalled on a scale of 0 (clearly normal) to 100 (clearly abnormal). The dataset included five categories of mammograms, with each category containing 40 cases. The categories were Cancer (current cancer-containing mammograms), Prior-Vis (prior mammograms with visible cancer signs), Contra (current âñormal' mammograms contralateral to the cancer), Prior-Invis (priors without visible cancer signs), and Normal (priors of normal cases). For each radiologist, four pairs of analyses were performed to evaluate whether the radiologists could distinguish mammograms in each category from the normal mammograms: Cancer vs Normal, Prior-Vis vs Normal, Contra vs Normal, and Prior-Invis vs Normal. The Area under Receiver Operating Characteristic curves (AUC) was calculated for each paired grouping and each radiologist. Wilcoxon Signed Rank test showed the AUC values were above-chance for all comparisons: Cancer (z=4.20, P<0.001); Prior-Vis (z=4.11, P<0.001); Contra (z=4.17, P<0.001); Prior-Invis (z=3.71, P<0.001). The results suggest that radiologists can distinguish patients who were diagnosed with cancer from individuals without breast cancer at an above-chance level based on a half-second glimpse of mammogram even before the lesion becomes apparently visible (Prior-Invis). Apparently, something about the breast parenchyma can look abnormal before the appearance of a localized lesion

Global processing provides malignancy evidence complementary to the information captured by humans or machines following detailed mammogram inspection

The information captured by the gist signal, which refers to radiologists’ first impression arising from an initial global image processing, is poorly understood. We examined whether the gist signal can provide complementary information to data captured by radiologists (experiment 1), or computer algorithms (experiment 2) based on detailed mammogram inspection. In the first experiment, 19 radiologists assessed a case set twice, once based on a half-second image presentation (i.e., gist signal) and once in the usual viewing condition. Their performances in two viewing conditions were compared using repeated measure correlation (rm-corr). The cancer cases (19 cases × 19 readers) exhibited non-significant trend with rm-corr = 0.012 (p = 0.82, CI: −0.09, 0.12). For normal cases (41 cases × 19 readers), a weak correlation of rm-corr = 0.238 (p < 0.001, CI: 0.17, 0.30) was found. In the second experiment, we combined the abnormality score from a state-of-the-art deep learning-based tool (DL) with the radiological gist signal using a support vector machine (SVM). To obtain the gist signal, 53 radiologists assessed images based on half-second image presentation. The SVM performance for each radiologist and an average reader, whose gist responses were the mean abnormality scores given by all 53 readers to each image was assessed using leave-one-out cross-validation. For the average reader, the AUC for gist, DL, and the SVM, were 0.76 (CI: 0.62–0.86), 0.79 (CI: 0.63–0.89), and 0.88 (CI: 0.79–0.94). For all readers with a gist AUC significantly better than chance-level, the SVM outperformed DL. The gist signal provided malignancy evidence with no or weak associations with the information captured by humans in normal radiologic reporting, which involves detailed mammogram inspection. Adding gist signal to a state-of-the-art deep learning-based tool improved its performance for the breast cancer detection

Computer-based image analysis in breast pathology

Whole slide imaging (WSI) has the potential to be utilized in telepathology, teleconsultation, quality assurance, clinical education, and digital image analysis to aid pathologists. In this paper, the potential added benefits of computer-assisted image analysis in breast pathology are reviewed and discussed. One of the major advantages of WSI systems is the possibility of doing computer-based image analysis on the digital slides. The purpose of computer-assisted analysis of breast virtual slides can be (i) segmentation of desired regions or objects such as diagnostically relevant areas, epithelial nuclei, lymphocyte cells, tubules, and mitotic figures, (ii) classification of breast slides based on breast cancer (BCa) grades, the invasive potential of tumors, or cancer subtypes, (iii) prognosis of BCa, or (iv) immunohistochemical quantification. While encouraging results have been achieved in this area, further progress is still required to make computer-based image analysis of breast virtual slides acceptable for clinical practice

Determining image processing features describing the appearance of challenging mitotic figures and miscounted nonmitotic objects

Context: Previous studies showed that the agreement among pathologists in recognition of mitoses in breast slides is fairly modest. Aims: Determining the significantly different quantitative features among easily identifiable mitoses, challenging mitoses, and miscounted nonmitoses within breast slides and identifying which color spaces capture the difference among groups better than others. Materials and Methods: The dataset contained 453 mitoses and 265 miscounted objects in breast slides. The mitoses were grouped into three categories based on the confidence degree of three pathologists who annotated them. The mitoses annotated as “probably a mitosis” by the majority of pathologists were considered as the challenging category. The miscounted objects were recognized as a mitosis or probably a mitosis by only one of the pathologists. The mitoses were segmented using k-means clustering, followed by morphological operations. Morphological, intensity-based, and textural features were extracted from the segmented area and also the image patch of 63 × 63 pixels in different channels of eight color spaces. Holistic features describing the mitoses' surrounding cells of each image were also extracted. Statistical Analysis Used: The Kruskal–Wallis H-test followed by the Tukey-Kramer test was used to identify significantly different features. Results: The results indicated that challenging mitoses were smaller and rounder compared to other mitoses. Among different features, the Gabor textural features differed more than others between challenging mitoses and the easily identifiable ones. Sizes of the non-mitoses were similar to easily identifiable mitoses, but nonmitoses were rounder. The intensity-based features from chromatin channels were the most discriminative features between the easily identifiable mitoses and the miscounted objects. Conclusions: Quantitative features can be used to describe the characteristics of challenging mitoses and miscounted nonmitotic objects

Global radiomic features from mammography for predicting difficult-to-interpret normal cases

This work aimed to investigate whether global radiomic features (GRFs) from mammograms can predict difficult-to-interpret normal cases (NCs). Assessments from 537 readers interpreting 239 normal mammograms were used to categorise cases as 120 difficult-to-interpret and 119 easy-to-interpret based on cases having the highest and lowest difficulty scores, respectively. Using lattice- and squared-based approaches, 34 handcrafted GRFs per image were extracted and normalised. Three classifiers were constructed: (i) CC and (ii) MLO using the GRFs from corresponding craniocaudal and mediolateral oblique images only, based on the random forest technique for distinguishing difficult- from easy-to-interpret NCs, and (iii) CC + MLO using the median predictive scores from both CC and MLO models. Useful GRFs for the CC and MLO models were recognised using a scree test. The CC and MLO models were trained and validated using the leave-one-out-cross-validation. The models’ performances were assessed by the AUC and compared using the DeLong test. A Kruskal–Wallis test was used to examine if the 34 GRFs differed between difficult- and easy-to-interpret NCs and if difficulty level based on the traditional breast density (BD) categories differed among 115 low-BD and 124 high-BD NCs. The CC + MLO model achieved higher performance (0.71 AUC) than the individual CC and MLO model alone (0.66 each), but statistically non-significant difference was found (all p > 0.05). Six GRFs were identified to be valuable in describing difficult-to-interpret NCs. Twenty features, when compared between difficult- and easy-to-interpret NCs, differed significantly (p < 0.05). No statistically significant difference was observed in difficulty between low- and high-BD NCs (p = 0.709). GRF mammographic analysis can predict difficult-to-interpret NCs

Global mammographic radiomic signature can predict radiologists' difficult-to-interpret normal cases

This study investigated whether a global radiomic signature (i.e., a set of global radiomic features) from mammograms can predict radiologists’ difficult-to-interpret normal cases. Retrospective non-identifiable data collected from 342 radiologists interpreting 81 normal mammograms were used to group cases as difficult-to-interpret (41 cases) and easy-to-interpret (40 cases) based on one-third of cases having the correspondingly highest and lowest difficulty scores. A set of 34 global radiomic features per image were extracted based on regions of interests delineated using lattice- and squared-based approaches, and normalised. Three machine learning classification models were constructed: 1). CC, using the 34 global radiomic features derived from craniocaudal images only, and 2). MLO, using the features from mediolateral oblique images only, both based on a random forest method for differentiating difficult-to-interpret from easy-to-interpret normal cases, and 3). CC+MLO model using the median predictive scores from both CC and MLO models. We trained and validated the models using leave-one-out-cross-validation approach. Performances of the models were measured by the area under the receiver operating characteristic curve (AUC). The CC+MLO model outperformed (0.73 AUC, 0.62 to 0.83) the CC (0.70 AUC, 0.62 to 0.78) and MLO (0.68 AUC, 0.60 to 0.76) models. The results showed that the global mammographic radiomic signature has the ability to predict radiologists’ difficult-to-interpret normal cases

Using Radiomics-Based Machine Learning to Create Targeted Test Sets to Improve Specific Mammography Reader Cohort Performance: A Feasibility Study

Mammography interpretation is challenging with high error rates. This study aims to reduce the errors in mammography reading by mapping diagnostic errors against global mammographic characteristics using a radiomics-based machine learning approach. A total of 36 radiologists from cohort A (n = 20) and cohort B (n = 16) read 60 high-density mammographic cases. Radiomic features were extracted from three regions of interest (ROIs), and random forest models were trained to predict diagnostic errors for each cohort. Performance was evaluated using sensitivity, specificity, accuracy, and AUC. The impact of ROI placement and normalization on prediction was investigated. Our approach successfully predicted both the false positive and false negative errors of both cohorts but did not consistently predict location errors. The errors produced by radiologists from cohort B were less predictable compared to those in cohort A. The performance of the models did not show significant improvement after feature normalization, despite the mammograms being produced by different vendors. Our novel radiomics-based machine learning pipeline focusing on global radiomic features could predict false positive and false negative errors. The proposed method can be used to develop group-tailored mammographic educational strategies to help improve future mammography reader performance